Recent work on Transformer-based large language models (LLMs) has revealed striking limits in their working memory capacity, similar to what has been found in human behavioral studies. Specifically, these models' performance drops significantly on N-back tasks as N increases. However, there is still a lack of mechanistic interpretability as to why this phenomenon would arise. Inspired by the executive attention theory from behavioral sciences, we hypothesize that the self-attention mechanism within Transformer-based models might be responsible for their working memory capacity limits. To test this hypothesis, we train vanilla decoder-only transformers to perform N-back tasks and find that attention scores gradually aggregate to the N-back positions over training, suggesting that the model masters the task by learning a strategy to pay attention to the relationship between the current position and the N-back position. Critically, we find that the total entropy of the attention score matrix increases as N increases, suggesting that the dispersion of attention scores might be the cause of the capacity limit observed in N-back tasks.

Working memory is a critical aspect of both human intelligence and artificial intelligence, serving as a workspace for the temporary storage and manipulation of information. In this paper, we systematically assess the working memory capacity of ChatGPT, a large language model developed by OpenAI, by examining its performance in verbal and spatial n-back tasks under various conditions. Our experiments reveal that ChatGPT has a working memory capacity limit strikingly similar to that of humans. Furthermore, we investigate the impact of different instruction strategies on ChatGPT's performance and observe that the fundamental patterns of a capacity limit persist. From our empirical findings, we propose that n-back tasks may serve as tools for benchmarking the working memory capacity of large language models and hold potential for informing future efforts aimed at enhancing AI working memory.

Near-infrared spectroscopy (NIRS) can measure neural activity through blood oxygenation changes in the brain in a wearable form factor, enabling unique applications for research in and outside the lab. NIRS has proven capable of measuring cognitive states such as mental workload, often using machine learning (ML) based brain-computer interfaces (BCIs). To date, NIRS research has largely relied on probes with under ten to several hundred channels, although recently a new class of wearable NIRS devices with thousands of channels has emerged. This poses unique challenges for ML classification, as NIRS is typically limited by few training trials which results in severely under-determined estimation problems. So far, it is not well understood how such high-resolution data is best leveraged in practical BCIs and whether state-of-the-art (SotA) or better performance can be achieved. To address these questions, we propose an ML strategy to classify working-memory load that relies on spatio-temporal regularization and transfer learning from other subjects in a combination that has not been used in previous NIRS BCIs. The approach can be interpreted as an end-to-end generalized linear model and allows for a high degree of interpretability using channel-level or cortical imaging approaches. We show that using the proposed methodology, it is possible to achieve SotA decoding performance with high-resolution NIRS data. We also replicated several SotA approaches on our dataset of 43 participants wearing a 3198 dual-channel NIRS device while performing the n-Back task and show that these existing methods struggle in the high-channel regime and are largely outperformed by the proposed method. Our approach helps establish high-channel NIRS devices as a viable platform for SotA BCI and opens new applications using this class of headset while also enabling high-resolution model imaging and interpretation.

Significance: We demonstrated the potential of using domain adaptation on functional Near-Infrared Spectroscopy (fNIRS) data to classify different levels of n-back tasks that involve working memory. Aim: Domain shift in fNIRS data is a challenge in the workload level alignment across different experiment sessions and subjects. In order to address this problem, two domain adaptation approaches -- Gromov-Wasserstein (G-W) and Fused Gromov-Wasserstein (FG-W) were used. Approach: Specifically, we used labeled data from one session or one subject to classify trials in another session (within the same subject) or another subject. We applied G-W for session-by-session alignment and FG-W for subject-by-subject alignment to fNIRS data acquired during different n-back task levels. We compared these approaches with three supervised methods: multi-class Support Vector Machine (SVM), Convolutional Neural Network (CNN), and Recurrent Neural Network (RNN). Results: In a sample of six subjects, G-W resulted in an alignment accuracy of 68 $\pm$ 4 % (weighted mean $\pm$ standard error) for session-by-session alignment, FG-W resulted in an alignment accuracy of 55 $\pm$ 2 % for subject-by-subject alignment. In each of these cases, 25 % accuracy represents chance. Alignment accuracy results from both G-W and FG-W are significantly greater than those from SVM, CNN and RNN. We also showed that removal of motion artifacts from the fNIRS data plays an important role in improving alignment performance. Conclusions: Domain adaptation has potential for session-by-session and subject-by-subject alignment of mental workload by using fNIRS data.